Lecture 05

Bill Perry

Lecture 4: Review

  • Introduction to hypothesis testing
  • The standard normal distribution
  • Standard error
  • Confidence intervals
  • Student’s t-distribution
  • H testing sequence
  • p-values

Our last graphs

Lecture 5: Overview

The objectives:

  • p-values
  • Brief review
  • H test for a single population
  • 1- and 2-sided tests
  • Hypothesis tests for two populations
  • Assumptions of parametric tests

asdfasfasd

Lecture 5: Statistical hypothesis testing

  • Major goal of statistics:
    • inferences about populations from samples…
      • assign degree of confidence to inferences
    • Statistical hypothesis testing:
      • formalized approach to inference
    • Hypotheses ask whether samples come from populations with certain properties
    • Often interested in quewstions about population means
      • but other questions are of interst

asdfasfasd

Lecture 5: Statistical hypothesis testing

Useful hypotheses: - Rely on specifying - null hypothesis (Ho) - alternate hypothesis (Ha)

  • Ho is the hypothesis of “no effect”
    • two samples from population with same mean
    • sample is from population of mean = 0
  • Ha (research hypothesis)
    • is the opposite of the Ho
    • or predicts that there is an effect of x on y
    • but does NOT suggest a direction

asdfasfasd

Lecture 5: Statistical hypothesis testing

Together Ho and Ha encompass all possible outcomes: - For Example:
- Ho: µ=0, Ha: µ ≠ 0 - mean equals 0 or mean does not equal 0 - Ho: µ=3700, Ha: µ ≠ 3700 - mean equals 3700 or mean does not equal 3700 - Ho: µ1 = µ2, Ha: µ1 ≠ µ2 - mean of population 1 equals mean of population 2 or it does not - Ho: µ > 0, Ha: µ ≤ 0 - can be directional mean is greater than 0 or mean is not equal or less than 0

asdfasfasd

Lecture 5: Statistical hypothesis testing

Tests assess likelihood of the null hypothesis being true

  • If the Ho is likely false, then Ha assumed to be correct
  • More precisely:
    • the long run probability of obtaining sample value (or more extreme one) if the null hypothesis is true
      • p(data|Ho) - the probability of observing the data given that the null hypothesis Ho is true

asdfasfasd

Lecture 5: Statistical hypothesis testing

Hypothesis tests

  • Expressed as p-value (0 to 1)
  • Interpret p-value as:
    • probability of obtaining sample value of statistic (or more extreme one) if Ho is true
  • High p-value:
    • high probability of obtaining sample statistic under Ho
      • if the null hypothesis (Ho) were true, you would frequently observe data similar to or more extreme than your sample statistic
      • your observed results are quite compatible with what the null hypothesis predicts
    • low p-value: low probability of obtaining sample statistic under Ho
      • if the null hypothesis (Ho) were true, you would rarely observe data similar to or more extreme than your sample statistic
      • Your results are unusual under the null hypothesis, suggesting that either you’ve witnessed a rare event or the null hypothesis may be incorrect

asdfasfasd

Lecture 5: Statistical hypothesis testing

Statistical test results: - p = 0.3 means that if I repeated the study 100 times, I would get this (or more extreme) result due to chance 30 times - p = 0.03 means that if I repeated the study 100 times, I would get this (or more extreme) result due to chance 3 times

Which p-value suggests Ho likely false?

asdfasfasd

Lecture 5: Statistical hypothesis testing

Statistical test results:

At what point reject Ho? - p < 0.05 conventional “significance threshold” (α = alpha or p value) - p < 0.05 means: - if Ho is true and we repeated the study 100 times - we would get this (or more extreme) result less than 5 times due to chance

asdfasfasd

Lecture 5: Statistical hypothesis testing

Statistical test results: - α is the rate at which we will reject a true null hypothesis (Type I error rate) - Lowering α will lower likelihood of incorrectly rejecting a true null hypothesis (e.g., 0.01, 0.001)

*Both Hs and α are specified **BEFORE collection of data and analysis*

asdfasfasd

Lecture 5: Statistical hypothesis testing

Traditionally α=0.05 is used as a cut off for rejecting null hypothesis

There is nothing magical about 0.05 - actual p-values need to be reported - also need to decide prior to study

p-value range Interpretation
P > 0.10 No evidence against Ho - data appear consistent with Ho
0.05 < P < 0.10 Weak evidence against the Ho in favor of Ha
0.01 < P < 0.05 Moderate evidence against Ho in favor of Ha
0.001 < P < 0.01 Strong evidence against Ho in favor of Ha
P < 0.001 Very strong evidence against Ho in favor of Ha

Lecture 5: Statistical hypothesis testing

Lecture 5: Statistical hypothesis testing

Fisher:

p-value as informal measure of discrepancy betwen data and Ho

“If p is between 0.1 and 0.9 there is certainly no reason to suspect the hypothesis tested. If it is below 0.02 it is strongly indicated that the hypothesis fails to account for the whole of the facts. We shall not often be astray if we draw a conventional line at .05 …”

Lecture 5: Statistical hypothesis testing

General procedure for H testing:

  • Specify Null (Ho) and alternate (Ha)
  • Determine test (and test statistic) to be used
  • Test statistic is used to compare your data to expectation under Ho (null hypothesis)
  • Specify significance (α or p value) level below which Ho will be rejected

asdfasfasd

Lecture 5: Statistical hypothesis testing

General procedure for H testing: - Collect data - Perform test - If p-value < α, conclude Ho is likely false and reject it - If p-value > α, conclude no evidence Ho is false and retain it

asdfasfasd

Lecture 5: Brief review

Recall… - Major goal of statistics: inferences about populations from samples… and assign degree of confidence to inferences - Statistical H-testing: formalized approach to inference - Relies on specifying null hypothesis (Ho) and alternate hypothesis (Ha) - Tests assess likelihood of the null hypothesis being true - Expressed as p-value: probability of obtaining sample value of statistic (or more extreme one) if Ho is true

asdfasfasd

Lecture 5: Brief review

Recall hospital example - Probability of getting sample like A (with ȳ at least as far away from 3700 as 3500)? - p(ȳ ≤ 3500 or ȳ ≥ 3900)

  • What about - 1-tailed or 2-tailed test?

  • Can solve using SND and z-scores

asdfasfasd

Lecture 5: Brief review

  • z= (3500-3700)/410 = -0.48

    • From z table: p= 0.3156 X 2
    • p of getting sample as far away from µ as A is = 0.6312 (63.1%)

  • But- usually can’t use z!

  • Can use t-distribution instead…

Lecture 5: H test for a single population

Let’s rephrase the question somewhat: What is probability that sample A (ȳ=3500, s=676) is from population with µ = 3700 (and σ = ?)?

Lecture 5: H test for a single population

This is a 1-sample test - 1-sample tests ask whether - µ is different than some number (e.g., 0 or 3700 or whatever) - based on sample - Another way: - probability this sample came from population with certain µ (mean) (e.g., 3700)

Lecture 5: H test for a single population

Common approach is one-sample t-test, which uses t statistic:
- St is value of statistic of interest from sample (e.g., ȳ) - θ is population value if Ho is true (e.g., µ=0) - SSt is standard error of the mean (s/√n)

What does equation tell us about relationships between variables?

\(t_s = \frac{St - \theta}{S_{St}}\)

test statistic (ts) is calculated by taking difference between observed statistic and value you’d expect under Ho then dividing by the standard error.

This standardization allows you to measure how many “standard errors away” your result is from what would be expected if the null hypothesis were true.

Lecture 5: H test for a single population

Process:

  • Specify Ho (e.g., µ = 0) and HA (e.g., µ ≠ 0)

  • Specify significance level (e.g., α = 0.05)

  • Take sample from population

  • Calculate:

\(t = \frac{\bar{y} - 0}{s/\sqrt{n}}\)

Lecture 5: H test for a single population

If sample from pop with mean or µ = 0 then t will be close to 0

|Large| t values are more likely if Ho false

Compare t with t-distribution: - if t is further than t at specified significance level (p = 0.05) (σ) - t has less than 5% chance of coming from null distribution

\(t_s = \frac{St - \theta}{S_{St}}\)

Lecture 5: H test for a single population

Is birth weight in population significantly different than 3700 g? - Specify Ho: µ = 3700 and Ha: µ ≠ 3700 - Specify significance level (α = 0.05) - Take sample from population: ȳ = 3500, s = 676

  • “What is probability of getting sample at least as far from µ as 3500?”

asdfasfasd

Lecture 5: H test for a single population

Is this one-tailed or two-tailed question?

  • Calculate t = (3500 – 3700) / (676/ √ 20)

  • t = -1.32

  • Compare with t-distribution for df=19

Lecture 5: H test for a single population

  • ~0.10 of t distribution left of t=-1.32 and ~0.1 right of 1.32 (~0.2 overall)

  • Probability of getting a sample as extreme (or more) as this is ~0.2

Lecture 5: H test for a single population

  • (p=0.203 to be exact)
  • No sufficient evidence to reject Ho

Lecture 5: H test for a single population

asdfa

asdfasfasd

Lecture 5: H test for a single population

asdfa

asdfasfasd

Lecture 5: H test for a single population

asdfa

asdfasfasd

Lecture 5: H test for a single population

asdfa

asdfasfasd

Lecture 5: H test for a single population

asdfa

asdfasfasd

Lecture 5: H test for a single population

asdfa

asdfasfasd

Lecture 5: H test for a single population

asdfa

asdfasfasd

Lecture 5: H test for a single population

asdfa

asdfasfasd

Lecture 5: H test for a single population

asdfa

asdfasfasd

Lecture 5: H test for a single population

asdfa

asdfasfasd

Lecture 5: H test for a single population

asdfa

asdfasfasd

Lecture 5: H test for a single population

asdfa

asdfasfasd

Lecture 5: H test for a single population

asdfa

asdfasfasd

Lecture 5: H test for a single population

asdfa

asdfasfasd

Lecture 5: H test for a single population

asdfa

asdfasfasd

Lecture 5: H test for a single population

asdfa

asdfasfasd